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ABSTRACT 

IBM has a long history in the application of formal methods to software development 
and verification. There have been many successes in the development of methods, tools, and 
training to support formal methods. And formal methods have been very successful on several 
projects. However, the use of formal methods has not been as widespread as hoped. This 
presentation summarizes several approaches that have been taken to encourage more 
widespread use of formal methods, and discusses the results so far. The basic problem is one 
of technology transfer, which is a very difficult problem. It is even more difficult for formal 
methods. General problems of technology transfer, especially the transfer of formal methods 
technology, are also discussed. Finally, some prospects for the future are mentioned. 
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Introduction and Purpose 


• To cover 


Oct/92 

Harlan Mills and SEW 


1. Some IBM involvement in Formal Methods (FM) 
projects 

2. A perspective on difficulties of technology transfer 
(beyond a single project) 


• Mills led massive software engineering education 
program 

- Software Engineering Workshop was cornerstone 
| 2 week course 

| Taught to all programmers 
| Required to pass final exam 


• Purpose is not to 

- sell the 'IBM approach" 

— argue against feasibility of FM 

• Purpose is to 

— learn from other FM technology transfer projects 

- suggest some possible future directions 


• SEW centered on mathematically-based verification 
— Functional instead of axiomatic 

| model oriented instead of property oriented 
| designed to scale up (stepwise refinement) 

| easier for programmers to understand 
- 2 pieces 

1. Deriving program functions 

| Trace tables (basically manual symbolic 
execution) 

| Recurson instead of loop invariants 

2 . Mod u le-orien ted 

| abstract data types 

| constraints/closure on state data (abstract 
state machine) 
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Harlan Mills and SEW ... 


• SEW designed to be practical 

- relatively informal 

- scaled up via abstraction/refinement 

- lots of examples and exercises 

- final test : pass/fail 

• Advocated for all programming, not just critical parts 

• no support beyond education 

- no tools 

- no consulting 

• General reaction was that it was impractical 

- too tedious 

- seemed only for toy problems 

• Did not gain widespread use 


Cleanroom 


• Named after silicon chip manufacturing environment 


• Built on SEW foundation, adding 


- Continuous inspections (SEW style verification) 


- Statisical testing (MTTF prediction) 


• Advertised through case studies, not classes 


Demonstration projects using highly skilled 
developers 


- To demonstrate benefits 


- To show it can be done, it is practical 


• Demonstrations projects were success stories 
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"'Cleanroom ... 


SEDL 


• Showcase project was COBOL/SF 

— Transforms unstructured COBOL into structured 
COBOL 

- 52,000 SLOCS developed using Cleanroom 

- Results 

| 740 SLOCS f labor month 

| 3.4 errors / KSLOC (before first execution) (70 

avg incl. UT) 

| no error ever found during operational use 


• Advocacy of Cleanroom continues 
— Widespread use not yet attained 
- But there is a lot of interest in Cleanroom 


• Intended to support SEW/Cleanroom verification 
concepts 

• Built as an extension to Ada 

• SEDL compiler generates Ada 

• Supports design execution 

- though SEDL generated code my be inefficient 

• Includes 

- Abstract data types (set, list, map, etc.) 

- User defined data models 

| model vs. representation 
| constraints 

- Supports mathematical notation 

| {X in CHARACTER : x / = 'Q'} 

| exists X in S : P(X) and exists Y in T : P(Y) 

| P>1 and not (exists Q in 2.. P-1 : P rem Q = 0) 

• Use of SEDL is not widespread 
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Stepwise Verification 

Stepwise Verification ... 

• Goal: increase use of more formal verification 

• Step 2: Contact specific projects 

- Build on SEW, Cleanroom foundation 

- Demo simple editing tools (support specs) 

- Investigate tools that support SEW 

- Demo on actual code from the project 

- Help projects get started 

- Discuss methodology 


- Motivate use 

• Step 1: Understand why SEW approach not widely used 

- Offer follow-up consulting 

— Survey of developers 


— Interviewed to those who participated in 

• Results 

Cleanroom pilots 



- Very positive results on one tool (SEED) 

- Results 



- Negative results on the methodology 

| Generally inconclusive, no primary reason(s) 


found 

| redundant work 

| Some themes were: 

| incompatible with current methodology 

1. Lack of experience 


2. Lack of support 

| impractical 

3. SEW reputation 
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TOP (Verification of ESs) 

• CICS is '60s vintage IBM product (assembler) 

• Some concern existed about verifiability of Expert 

transferred to Hursley, U.K. 

Systems (ESs) 

• Hursley began big program towards more formal S/W 

• Study of the problem pointed to one area: Specification 

Eng. 

— Poor low level languages 


- Almost no design 

• Key approach was formal specs using Z 



• Led to development of an ES design language 

• Worked hand-in-hand with PRG at Oxford 

— Based on work done at USC ISI (LOOM and 


CLASP) 

• Began with selected modules and later expanded use 

| higher level language (term subsumption + 


OOP) 

• Results 

— we added annotations 

— Some initial "culture shock" for both parties 

| Pre and post conditions 

— Now some 50 people work directly with Z specs 

| Global constraints 

— Very positive qualitative results (people like it) 

| etc. 

— Limited quantitative data indicates 

- Supported by a compiler (a la SEDL) 

| earlier error removal 

- Supports modularity (a la Ada) 

| fewer inserted errors 

- Supports annotations 

| slight cost reduction (9%) 



• Cleanroom has also been extended to expert systems 

• Use of Z continues at Hursley, but very few other 


places 

• Neither approach has gained widespread use 
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Other Projects and Approaches 


Application above the code level 

- Development of a "Box Structures" design 
language 

- Development of a "Box Structures" approach to 
requirements 

- Results 

| SA/SD approach to design most popular new 
approach 

| Requirements still written in English 

• Emphasis on SEW concepts 

— Concepts generally well accepted 
— Loss of rigor reduces mathematical basis 


Note on Quality Emphasis 

• Software quality has extreme emphasis 

- Great emphasis on process improvement 

- Serious attention given to quality goals and 
measurement 

— Quality motivation programs 
| awards and recognition 
| Manned Flight Awareness program 

• There is willingness to work hard and invest for quality 

• The question is not what or how much but how 
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- FM is generally perceived as not helping 
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^Summary 


• Goal was to increase the use of formal mathematical 
approaches to software development (beyond a single 
project) 

1. First through education 

2. Then through demonstration projects 

3. Then through tool support 

4. Then by making methods more practical 

5. Finally through direct support (consulting) 

• There have been successes 

- not nearly as widespread as desired 

• This story is not unique to FM 


Conclusions 


• Conclusion: Technology Transfer is very hard, even 
with 

- extensive education 

- tools support 

- demonstrated successes 

• Possible future directions 

- More consulting ("hand holding") (product 
champions) 

- Use only a core group (FM may just not be for 
everybody) 

- Require use of FM (selected groups) 

— Success story close to home 

| technology transfer diminishes rapidly as a 
function of distance 

| long term committment is required (success 
story requires continued follow-up) 

- Different formal method(s) 

- Different tools (e g., theorem prover) 


- The problem is with technology transfer, not with 
technology 
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